Processing method for sound effect of recording and mobile terminal

ABSTRACT

A method of processing for sound effect of recording and a mobile terminal are disclosed in embodiments of the present disclosure. The method includes recording N audio signals generated from N sound sources, N being an integer greater than or equal to 2; obtaining location relationships of the N sound sources, determining a scene where the N sound sources are located in, and determining a reverberation algorithm according to the location relationships of the N sound sources and the scene where the N sound sources are located in; and performing a sound effect processing for the N audio signals according to the reverberation algorithm to obtain a reverbed sound effect.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to Chinese Patent Application No.201811438046.6, filed on Nov. 27, 2018, the content of which is hereinincorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure generally relates to the technical field ofaudio, and more particularly, to a processing method for sound effect ofrecording, a mobile terminal, and a non-transitory storage medium.

BACKGROUND

As the mobile terminals (such as mobile phones, tablets, etc.) are usedwidely, the mobile terminals can support more and more applications, ofwhich the functions are more and more powerful. The mobile terminals aredeveloping in a diversified and personalized direction, becomingindispensable electronic supplies in users' lives. Currently, recordingis a necessary function for the mobile terminal.

SUMMARY

According to one aspect of the present disclosure, a processing methodfor sound effect of recording is provided, which includes: recording Naudio signals generated from N sound sources, N being an integer greaterthan or equal to 2; obtaining location relationships of the N soundsources, determining a scene where the N sound sources are located in,and determining a reverberation algorithm according to the locationrelationships of the N sound sources and the scene where the N soundsources are located in; and performing a sound effect processing for theN audio signals according to the reverberation algorithm to obtain areverbed sound effect.

According to another aspect of the present disclosure, a mobile terminalis provided, which includes a processor and a non-transitory memorystoring one or more programs, wherein the one or more programs areconfigured to be executed by the processor to perform a method, and themethod includes: recording N audio signals generated from N soundsources, N being an integer greater than or equal to 2; obtaininglocation relationships of the N sound sources, determining a scene wherethe N sound sources are located in, and determining a reverberationalgorithm according to the location relationships of the N sound sourcesand the scene where the N sound sources are located in; and performing asound effect processing for the N audio signals according to thereverberation algorithm to obtain a reverbed sound effect.

According to yet another aspect of the present disclosure, anon-transitory storage medium is provided, which stores one or moreprograms, when executed, causing a processor to perform a processingmethod, wherein the processing method includes: recording N audiosignals generated from N sound sources, N being an integer greater thanor equal to 2; obtaining location relationships of the N sound sources,determining a scene where the N sound sources are located in, anddetermining a reverberation algorithm according to the locationrelationships of the N sound sources and the scene where the N soundsources are located in; and performing a sound effect processing for theN audio signals according to the reverberation algorithm to obtain areverbed sound effect.

BRIEF DESCRIPTION OF DRAWINGS

In order to make the technical solution described in the embodiments ofthe present disclosure or the related art more clearly, the drawingsused for the description of the embodiments or the related art will bebriefly described. Apparently, the drawings described below are onlysome embodiments of the present disclosure. It should be understoodthat, one skilled in the art may acquire other drawings based on thesedrawings, without making any inventive work.

FIG. 1 is a flow chart of a processing method for sound effect ofrecording according to some embodiments of the present disclosure;

FIG. 2 is a diagram of a scene of multi-person video conferenceaccording to some embodiments of the present disclosure;

FIG. 3 is a diagram of analog transmission of an audio signal accordingto some embodiments of the present disclosure;

FIG. 4 is a flow chart of another processing method for sound effect ofrecording according to some embodiments of the present disclosure;

FIG. 5 is a schematic structural diagram of a processing apparatus forsound effect of recording according to some embodiments of the presentdisclosure;

FIG. 6 is a schematic structural diagram of a mobile terminal accordingto some embodiments of the present disclosure;

FIG. 7 is a schematic structural diagram of another mobile terminalaccording to some embodiments of the present disclosure.

DETAILED DESCRIPTION

In order to better understand the present disclosure for those skilledin the art, the technical solutions in the embodiments of the presentdisclosure are clearly and completely described in conjunction with thedrawings in the embodiments of the present disclosure. It is obviousthat the described embodiments are only a part of the embodiments of thepresent disclosure, and not all embodiments. All other embodimentsobtained by the ordinary skilled in the art based on the embodiments inthe present disclosure without the creative work are all within thescope of the present disclosure.

The terms “first”, “second” and the like in the specification and claimsof the present disclosure and the above drawings are used to distinguishdifferent objects, and are not intended to describe a specific order.Furthermore, the terms “including” and “having”, and any modificationthereof are intended to cover unexclusive inclusion. For example, aprocess, method, system, product, or device that includes a series ofsteps or units is not limited to the listed steps or units, butoptionally also includes steps or units not listed, or alternativelyincludes other steps or units inherent to these processes, methods,products, or equipment.

“Embodiment” herein means that a particular feature, structure, orcharacteristic described with reference to embodiments can be includedin at least one embodiment of the present disclosure. The term appearingin various places in the specification are not necessarily referring tothe same embodiment, and are not exclusive or alternative embodimentsthat are mutually exclusive with other embodiments. Those skilled in theart will understand explicitly and implicitly that the embodimentsdescribed herein can be combined with other embodiments.

A processing method for sound effect of recording is provided, whichincludes: recording N audio signals generated from N sound sources, Nbeing an integer greater than or equal to 2; obtaining locationrelationships of the N sound sources, determining a scene where the Nsound sources are located in, and determining a reverberation algorithmaccording to the location relationships of the N sound sources and thescene where the N sound sources are located in; and performing a soundeffect processing for the N audio signals according to the reverberationalgorithm to obtain a reverbed sound effect.

In one embodiment, the recording N audio signals generated from N soundsources includes: recording the N audio signals generated from N soundsources through N microphones each of which is arranged adjacent to arespective one of the N sound sources.

In one embodiment, the obtaining location relationships of the N soundsources includes: measuring a distance and a relative orientationbetween any two of the N microphones through a measuring device.

In one embodiment, the determining a reverberation algorithm accordingto the location relationships of the N sound sources and the scene wherethe N sound sources are located in includes: determining thereverberation algorithm according to the distance and relativeorientation between any two of the N microphones and the scene where theN sound sources are located in.

In one embodiment, the method is applied to a video call terminal; andthe obtaining location relationships of the N sound sources includes:obtaining a first location of each of the N sound sources with respectto the video call terminal.

In one embodiment, the determining a reverberation algorithm accordingto the location relationships of the N sound sources and the scene wherethe N sound sources are located in includes: determining thereverberation algorithm according to the first location of the each ofthe N sound sources with respect to the video call terminal and thescene where the N sound sources are located in.

In one embodiment, the scene where the N sound sources are located inincludes an indoor scene; and the determining a reverberation algorithmaccording to the location relationships of the N sound sources and thescene where the N sound sources are located in includes: determiningindoor parameters of the indoor scene, wherein the indoor parametersinclude a parameter of indoor area, a parameter of indoor volume, aparameter of indoor material, and a parameter of indoor air medium;determining reverberation parameters according to the indoor parametersand the location relationships of the N sound sources, wherein thereverberation parameters include a ratio of direct sounds, attenuationcoefficients corresponding to the direct sounds, a radio of echoes,attenuation coefficients corresponding to the echoes, and areverberation frequency range; and determining the reverberationalgorithm according to the reverberation parameters.

In one embodiment, the performing a sound effect processing for the Naudio signals according to the reverberation algorithm to obtain areverbed sound effect includes: performing a denoising processing forthe N audio signals to obtain the denoised N audio signals through amulti-noise filtering engine; and performing the sound effect processingfor the denoised N audio signals according to the reverberationalgorithm to obtain the reverbed sound effect.

In one embodiment, the method further includes transmitting the reverbedsound effect to a video call terminal to play the reverbed sound effect.

A mobile terminal is provided, which includes a processor and anon-transitory memory storing one or more programs, wherein the one ormore programs are configured to be executed by the processor to performa method, and the method includes: recording N audio signals generatedfrom N sound sources, N being an integer greater than or equal to 2;obtaining location relationships of the N sound sources, determining ascene where the N sound sources are located in, and determining areverberation algorithm according to the location relationships of the Nsound sources and the scene where the N sound sources are located in;and performing a sound effect processing for the N audio signalsaccording to the reverberation algorithm to obtain a reverbed soundeffect.

In one embodiment, each of the N audio signals are recorded through arespective one of N microphones, and each of the N microphones isarranged adjacent to a respective one of the N sound sources.

In one embodiment, the location relationships of the N sound sourcesinclude a distance and a relative orientation between any two of the Nmicrophones.

In one embodiment, the reverberation algorithm is determined accordingto the distance and the relative orientation between any two of the Nmicrophones and the scene where the N sound sources are located in.

In one embodiment, the location relationships of the N sound sourcesinclude a first location of each of the N sound sources with respect tothe mobile terminal.

In one embodiment, the reverberation algorithm is determined accordingto the first location of the each of the N sound sources with respect tothe video call terminal and the scene where the N sound sources arelocated in.

In one embodiment, the scene where the N sound sources are located inincludes an indoor scene; and the reverberation algorithm is determinedaccording to reverberation parameters, wherein the reverberationparameters is determined according to indoor parameters of the indoorscene and the location relationships of the N sound sources and includea ratio of direct sounds, attenuation coefficients corresponding to thedirect sounds, a radio of echoes, attenuation coefficients correspondingto the echoes, and a reverberation frequency range, and the indoorparameters include a parameter of indoor area, a parameter of indoorvolume, a parameter of indoor material, and a parameter of indoor airmedium.

A non-transitory storage medium is provided, which stores one or moreprograms, when executed, causing a processor to perform a processingmethod, wherein the processing method includes: recording N audiosignals generated from N sound sources, N being an integer greater thanor equal to 2; obtaining location relationships of the N sound sources,determining a scene where the N sound sources are located in, anddetermining a reverberation algorithm according to the locationrelationships of the N sound sources and the scene where the N soundsources are located in; and performing a sound effect processing for theN audio signals according to the reverberation algorithm to obtain areverbed sound effect.

In one embodiment, each of the N audio signals are recorded through arespective one of N microphones, and each of the N microphones isarranged adjacent to a respective one of the N sound sources.

In one embodiment, the location relationships of the N sound sourcesinclude a distance and a relative orientation between any two of the Nmicrophones.

In one embodiment, the scene where the N sound sources are located inincludes an indoor scene; and the reverberation algorithm is determinedaccording to reverberation parameters, wherein the reverberationparameters is determined according to indoor parameters of the indoorscene and the location relationships of the N sound sources and includea ratio of direct sounds, attenuation coefficients corresponding to thedirect sounds, a radio of echoes, attenuation coefficients correspondingto the echoes, and a reverberation frequency range, and the indoorparameters include a parameter of indoor area, a parameter of indoorvolume, a parameter of indoor material, and a parameter of indoor airmedium.

The mobile terminal involved in the embodiments of the presentdisclosure may include various handheld devices, wireless devices,wearable devices, computing devices, or other processing devicesconnected to the wireless modem, which have wireless communicationfunctions, and various forms of user equipment (UE), a mobile station(MS), the terminal device and the like. For convenience of description,the devices mentioned above are collectively referred to as mobileterminals.

The embodiments of the present disclosure are described in detail below.

FIG. 1 is a flow chart of a processing method for sound effect ofrecording according to some embodiments of the present disclosure. Theprocessing method may be executed by a mobile terminal described above.As shown in FIG. 1, the processing method for sound effect of recordingincludes actions/operations in the following blocks.

At block 101, the method records N audio signals generated from N soundsources, N being an integer greater than or equal to 2.

The processing method for sound effect according to embodiments of thepresent disclosure may be applied to a scene for recording multi-personaudio, which is a scene where there are multiple sound sources torespectively emit a sound during the recording, for example, a scene ofmulti-person video conference. That is, in the scene of multi-personvideo conference, each sound source emits an audio signal during therecording, such that N audio signals are recorded.

N sound sources may refer to N persons in the process of recording. Themobile terminal records N audio signals generated from N sound sourcesthrough N microphones. Specifically, the mobile terminal separatelyrecords N audio signals generated from N sound sources, which meansspecifically that the mobile terminal records analog audio signals of Nsound sources through N microphones, and performs analog conversion forthe analog audio signals of the N sound sources according to a certainsampling frequency (for example, 22.05 kHz, 44.1 KHz, 48 KHz) to obtainthe N audio signals generated from N sound sources. That is, each of Naudio signal obtained by analog conversion means digital data.

Alternatively, block 101 may include actions in the following.

The method records N audio signals generated from N sound sourcesthrough N microphones each of which is arranged adjacent to a respectiveone of the N sound sources.

In one embodiment of the present disclosure embodiment, N microphonesclose to N sound sources, respectively. that is, each of N microphonescloses to a respective sound source. The N microphones are configuredfor collecting sounds, which N sound sources emit, to form N audiosignals. The data of audio signal may be stored in the mobile terminalin a certain audio format (for example, wave format, mp3 format, wmaformat). The mobile terminal may be in wired connection with the Nmicrophones, and may also in wireless connection (for example, aBluetooth connection). After N microphones collect N audio signals, theN audio signals are transmitted to the mobile terminal, and then themobile terminal may perform a sound effect processing for the N audiosignals.

At block 102, the method obtains location relationships of the N soundsources, determines a scene where the N sound sources are located in,and determines a reverberation algorithm based on location relationshipsof the N sound sources and the scene where the N sound sources arelocated in.

In one embodiment of the present disclosure, in a scene of multi-personlive conference, the mobile terminal may obtain location relationshipsof the N sound sources, and determines the reverberation algorithm basedon the relative location relationship of the N sound sources. In a sceneof multi-person remote video conference, the mobile terminal may obtainlocation relationships of the N sound sources. The mobile terminalincludes a first camera. The first camera is a camera which takes apicture of the N sound sources. In one embodiment, the locationrelationships of N sound sources include a location of each of the Nsound sources with respect to the first camera. The reverberationalgorithm may be determined based on the locations of the N soundsources with respect to the first camera. In another embodiment, thelocation relationships of N sound sources include a location of each ofthe N sound sources with respect to the first camera and a distance andrelative orientation between any two of the N sound sources. Thereverberation algorithm may be determined based on the location of eachof the N sound sources with respect to the first camera and a distanceand relative orientation between any two of the N sound sources.

In one embodiment, the location relationships of the N sound sourcesinclude a distance between any two of the N sound sources, relativeorientation between any two of the N sound sources, and relativelocation relationship graph among the N sound sources. The reverberationalgorithm may be determined based on the distance and relativeorientation between any two of the N sound sources.

FIG. 2 is a diagram of a scene of multi-person video conferenceaccording to some embodiments of the present disclosure. As shown inFIGS. 2, 3, 4, and 6 in the conference room are participants,participants 3 and 4 are in the conference room, and participant 6 maybe out of the conference room, which means a remote video conference.Participant 6 may be in the conference room, which means a liveconference. 5 in the conference room refers to a microphone forrecording multi-channel sound. FIG. 2 includes a relative locationrelationship graph among participants 3, 4, and 6.

An example of a remote video conference will be described in thefollowing.

When the participant 6 is out of the conference room and is remoteaccessed, the relative distances between participants 3 and 6 andbetween the participants 4 and 6 can be identified through the camera 2,and sounds of the participants 3 and 4 are recorded through themicrophone 5. After sounds of the participant 3 and the participant 4are recorded through the microphone 5, the sounds will not directlyplayed to the participant 6, but a reverberation algorithm is determinedaccording to the distances and orientations between participant 3 andparticipant 6 and between the participant 4 and the participant 6 andparameters of the conference room (for example, volume, area, wallmaterial, floor material, ceiling material of the conference room), andthen the sounds of the participants 3 and 4 is processed according tothe reverberation algorithm to obtain a reverbed sound effect. When theprocessed reverbed sound effect is played, the participant 6 can hearthat the participant 3 is at left and the participant 4 is at right.Thus, the participant 6 who participates in the video conferenceremotely has the experience of participating in the conference on thespot, which improves sound effect of audio recorded in the remote videoconference.

Alternatively, at block 102, the mobile terminal acquires locationrelationships of N sound sources, which includes the following.

(11) the mobile terminal measures a distance and a relative orientationbetween any two of the N microphones through a measuring device.

(12) the mobile terminal determines the location relationships of the Nsound sources based on the distance and relative orientation between anytwo of the N microphones. That is, the location relationships of the Nsound sources include a distance and a relative orientation between anytwo of the N sound sources.

In one embodiment of the present disclosure, in order to maintain a goodsound receiving effect, a microphone is generally arranged adjacent to asound source, and the location relationship of the sound source inembodiments of the present disclosure may be refer to the locationrelationship of the microphone arranged adjacent to the sound source.

Alternatively, the N sound sources are located in an indoor scene. Atblock 102, the mobile terminal determines the reverberation algorithmbased on location relationships of the N sound sources and the scenewhere the N sound sources are located in, which includes the following.

(21) the mobile terminal determines indoor parameters of the indoorscene, and the indoor parameters include a parameter of indoor area, aparameter of indoor volume, a parameter of indoor material, and aparameter of indoor air medium.

(22) the mobile terminal determines reverberation parameters accordingto the indoor parameters and the location relationships of the N soundsources, and the reverberation parameters include a ratio of directsounds, attenuation coefficients corresponding to the direct sounds, aradio of echo, attenuation coefficients corresponding to the echoes, anda reverberation frequency range.

(23) the mobile terminal determines the reverberation algorithmaccording to the reverberation parameters.

In one embodiment of the present disclosure, the indoor scene may be anindoor scene for a conference room. As the indoor parameters aredifferent, corresponding reverberation parameters are also different. Amodel of the indoor scene may be established, and the indoor parametersof the indoor scene, the location relationships of the N sound sourceand relative location relationship between a virtual audio receiving endand the N sound source are input into the model to obtain thereverberation parameters. The virtual audio receiving end is a virtuallocation of the participant for remote video conference in the indoorscene, and the location of the virtual audio receiving end may be alocation of a camera for the remote video conference.

Sound emitted by each of N sound sources may be respectively performed asound effect processing with the reverberation algorithm, and then thosesounds are collectively played through N speaker to form the reverbedeffect.

At block 103, the method performs a sound effect processing the N audiosignals according to the reverberation algorithm to obtain a reverbedsound effect.

In one embodiment of the present disclosure, an audio playing end of theaudio recording scene (such as a conference room) plays audio. For theaudio receiving end, the received audio signal includes a reflectedsound signal after various complicated physical reflections, in additionto the direct sound signal directly transmitted from the audio playingend. The reflected sound signal arrives later than the direct soundsignal and its energy is attenuated due to physical reflection. Indifferent audio recording scenes, there will be a large difference inthe delay of the reflected sound and the energy attenuation, whereresult in different hearing feelings at the audio receiving end.Therefore, for different audio recording scenes, different reverberationalgorithms may be used for processing sound effect.

FIG. 3 is a diagram of analog transmission of an audio signal accordingto some embodiments of the present disclosure. The audio signalgenerated by the audio playing end in FIG. 3 can reach the audioreceiving end in a direct manner and a reflection manner, and then areverbed effect is formed at the audio receiving end. Two reflectionpaths are illustrated in FIG. 3, the first reflection path has tworeflection to the audio receiving end, and the second reflection pathhas one reflection to the audio receiving end. FIG. 3 only illustratesan example for audio signal transmission. The audio signal may bereflected once, twice, or more than two times to reach the audioreceiving end. For different audio recording scenes, the number ofreflections and reflection paths are different. Regardless of whetherthe audio signal is direct or reflected, it will be attenuated at acertain degree. The attenuation coefficient is determined by thedistance of the path, the number of reflections, the transmissionmedium, and the material of the reflection point.

In a scene for multi-person remote video conference, the reverberationalgorithm may be determined according to location relationships of the Nsound sources in the conference room in the scene for multi-personremote video conference and related parameters of the conference room(e.g., volume, area, wall material, floor material, ceiling material ofthe conference room). After N audio signals generated from the N soundsources are recorded, the N audio signals may be processed according tothe determined reverberation algorithm to obtain the reverbed soundeffect.

Alternatively, block 103 may specifically include the following.

(31) the method performs a denoising processing for the N audio signalsto obtain the denoised N audio signals through a multi-noise filteringengine.

(32) the method performs a sound effect processing for the denoised Naudio signals according to the reverberation algorithm to obtain thereverbed sound effect.

In embodiments of the present disclosure, before performing the soundeffect processing, the mobile terminal may use the multi-noise filteringengine to perform the denoising processing for the N audio signals.Thus, noise generated during the recording is removed to improverecording effect.

In embodiments of the present disclosure, during an audio recording, thereverberation algorithm may be determined according to locationrelationships of N sound sources, and a sound effect processing isperformed for the N audio signals generated from the N sound sourcesaccording to the determined reverberation algorithm. It is achieved thatmultiple audio signals of multiple sound sources are performed with asound effect processing. A sound effect processing can be performed formultiple audio signals during a recording, so as to improve sound effectof an audio recorded in a multi-person audio recording scene.

FIG. 4 a flow chart of another processing method for sound effect of arecording according to some embodiments of the present disclosure. FIG.4 is obtained from optimization based on FIG. 1. The method of FIG. 4 isalso executed by the mobile terminal. The mobile terminal is applied forvideo call. A video call system includes a first call terminal (i.e. themobile terminal) and a second call terminal. As shown in FIG. 4, theprocessing method includes actions/operations in the following blocks.

At block 401, the method records N audio signals generated from N soundsources, N being an integer greater than or equal to 2.

At block 402, the method obtains location relationships of the N soundsources, determines a scene where the N sound sources are located in,and determines a reverberation algorithm based on location relationshipsof the N sound sources and the scene where the N sound sources arelocated in.

At block 403, the method performs a sound effect processing for the Naudio signals according to the reverberation algorithm to obtain areverbed sound effect.

For the specific implementation of the blocks 401 to 403 in theembodiments of the present disclosure, reference may be made to theblocks 101 to 103 shown in FIG. 1, and details are not described hereinagain.

At block 404, the method transmits the reverbed sound effect to thesecond call terminal to play.

The method in embodiments of the present disclosure is applied to avideo call system, which includes at least two video call terminals (afirst call terminal and a second call terminal), a multi-noise filteringprocessing engine, and a multi-noise filtering application programminginterface (API) management server. The multi-noise filtering processingengine and the multi-noise filtering API management server are connectedto the two video call terminals through a communication network. When afar-field video call is performed between the first call terminal andthe second call terminal, far-end sound of a caller of the first callterminal and multiple noise sources are simultaneously received andrecorded by the first call terminal. The multi-noise filter processingengine filters the far-field sound and multiple noise sources. Themulti-noise filter API management server suppresses the multiple noisesources to enhances the sound of the caller. The second call terminalreceives the processed sound of the caller. The call quality of voicedata of a remote video call is improved in embodiments of the presentdisclosure improves.

The first call terminal may include a first camera, a plurality of firstmicrophones, a first speaker, a first sound effect processing module, afirst memory, and a first transceiver. A first camera is configured forcapturing an image of N sound sources. Each of the first microphones isconfigured for recording one audio signals generated from a respectiveone of the N sound sources, and the number of first microphones isgreater than or equal to N. The first sound effect processing module isconfigured for performing a sound effect processing for N audio signalsgenerated from the N sound sources to obtain a reverbed sound effect.The first memory is configured for storing N audio signals generatedfrom the N sound sources. The first transceiver is configured fortransmitting the reverbed sound effect to the second call terminal. Thefirst microphones may be a component of the first call terminal or adiscrete module. When the first microphone is a discrete module, thefirst microphone includes a transceiver, and the transceiver of thefirst microphone is connected to the first transceiver of the first callterminal in wire or wirelessly.

The second call terminal may include a second camera, a secondmicrophone, a plurality of second speakers, a second sound effectprocessing module, a second memory, and a second transceiver. The secondcall terminal and the first call terminal transmit audio data and soundeffect data through the first transceiver and the second transceiver.The number of the second speakers is equal to or greater than N. Thesecond call terminal may play the reverbed sound effect transmitted bythe first call terminal through the second speakers. Thus, the reverbedeffect is achieved.

Alternatively, at block 402, the method obtains location relationshipsof the N sound sources, which is the following specifically.

The mobile terminal obtains locations between the N sound sources andthe first call terminal. That is, the location relationship of a soundsource includes a location of the sound source with respect to the firstcall terminal.

At block 402, the mobile terminal determines the reverberation algorithmbased on location relationships of the N sound sources and the scenewhere the N sound sources are located in, which is the followingspecifically.

The mobile terminal determines the reverberation algorithm based onlocations between the N sound sources and the first call terminal andthe scene where the N sound sources are located in. That is, thereverberation algorithm is determined based on locations of the N soundsources with respect to the first call terminal and the scene where theN sound sources are located in.

In one embodiment, the mobile terminal may obtain location of each of Nsound sources with respect to the first call terminal.

In embodiments of the present disclosure, in the video call system, Naudio signals of the first call terminal may be processed to obtain thereverbed sound effect, and the first call terminal transmits thereverbed sound effect to the second call terminal for playing. Thus, thesecond call terminal can hear sound with the reverbed sound effect.Therefore, the call quality of the voice data of the remote video callis improved.

The above description mainly introduces the solution of the embodimentof the present disclosure from the perspective of the execution processof the method side. It can be understood that, in order to implement theabove functions, the mobile terminal includes corresponding hardwarestructures and/or software modules for performing various functions.Those skilled in the art will readily appreciate that the presentdisclosure can be implemented in hardware or a combination of hardwareand computer software in combination with the elements and algorithmsteps of the various examples described in the embodiments disclosedherein. Whether a function is implemented in hardware or a way ofdriving hardware by computer software depends on the specificapplication and design constraints of the solution. A person skilled inthe art can use different methods for implementing the describedfunctions for each particular application, but such implementationshould not be considered to be beyond the scope of the presentdisclosure.

The embodiments of the present disclosure may divide the mobile terminalinto function units according to the foregoing method examples. Forexample, each function unit may be divided according to each function,or two or more functions may be integrated into one processing unit. Theabove integrated unit can be implemented in the form of hardware or inthe form of a software function unit. It should be noted that thedivision of the unit in the embodiments of the present disclosure isschematic, and is only a division for logical function. In actualimplementation, there may be another division manner.

FIG. 5 is a schematic structural diagram of a processing apparatus forsound effect of recording according to some embodiments of the presentdisclosure. As shown in FIG. 5, the processing apparatus 500 includes arecording unit 501, an obtaining unit 502, a determining unit 503, and areverberation unit 504.

The recording unit 501 is configured for recording N audio signalsgenerated from N sound sources, N being an integer greater than or equalto 2.

The obtaining unit 502 is configured for acquiring locationrelationships of the N sound sources.

The determining unit 503 is configured for determining a scene where theN sound sources are located in and determining a reverberation algorithmbased on location relationships of the N sound sources and the scenewhere the N sound sources are located in.

The reverberation unit 504 is configured for performing a sound effectprocessing for the N audio signals according to the reverberationalgorithm to obtain a reverbed sound effect.

Alternatively, the recording unit 501 records N audio signals generatedfrom N sound sources, which is the following specifically.

Recording N audio signals generated from N sound sources through Nmicrophones each of which is arranged adjacent to a respective one ofthe N sound sources.

Alternatively, the obtaining unit 502 obtains location relationships ofthe N sound sources, which is the following specifically.

Measuring a distance and a relative orientation between any two of the Nmicrophones through a measuring device.

Determining the location relationships of the N sound sources based onthe distance and relative orientation between any two of the Nmicrophones.

Alternatively, the N sound sources are located in an indoor scene. Thedetermining unit 503 determines a reverberation algorithm based onlocation relationships of the N sound sources and the scene where the Nsound sources are located in, which includes the following specifically.

Determining indoor parameters of the indoor scene, and the indoorparameters include a parameter of indoor area, a parameter of indoorvolume, a parameter of indoor material, and a parameter of indoor airmedium.

Determining reverberation parameters according to the indoor parametersand the relative location relationship of the N sound sources, and thereverberation parameters include a ratio of direct sounds, attenuationcoefficients corresponding to the direct sounds, a radio of echoes,attenuation coefficients corresponding to the echoes, and areverberation frequency range.

Determining the reverberation algorithm according to the reverberationparameters.

Alternatively, the reverberation unit 504 performs a sound effectprocessing for the N audio signals according to the reverberationalgorithm to obtain a reverbed sound effect, which includes thefollowing specifically.

Performing a denoising processing for the N audio signals to obtain thedenoised N audio signals through a multi-noise filtering engine.

Performing the sound effect processing for the denoised N audio signalsaccording to the reverberation algorithm to obtain the reverbed soundeffect.

Alternatively, the processing apparatus 500 is applied to a video callsystem, which includes a first call terminal and a second call terminal.The processing apparatus 500 may further include a transmitting unit505.

The transmitting unit 505 is configured to transmit the reverbed soundeffect from the first call terminal to the second call terminal forplaying.

Alternatively, the obtaining unit 502 obtains location relationships ofthe N sound sources, which is the following specifically.

Obtaining locations between the N sound sources and the first callterminal.

The determining unit 503 determines the reverberation algorithm based onlocation relationships of the N sound sources and the scene where the Nsound sources are located in, which is the following specifically.

Determining the reverberation algorithm based on locations between the Nsound sources and the first call terminal and the scene where the Nsound sources are located in.

The second call terminal includes N speakers.

In the processing apparatus as shown in FIG. 5, during an audiorecording, the reverberation algorithm may be determined according torelative location relationship of N sound sources, and a sound effectprocessing is performed for the N audio signals generated from the Nsound sources according to the determined reverberation algorithm. It isachieved that multiple audio signals of multiple sound sources areperformed with a sound effect processing. A sound effect processing canbe performed for multiple audio signals during a recording, so as toimprove sound effect of an audio recorded in a multi-person audiorecording scene.

FIG. 6 is a schematic structural diagram of a mobile terminal accordingto some embodiments of the present disclosure. As shown in FIG. 6, themobile terminal 600 includes a processor 601 and a non-transitory memory602. The mobile terminal 600 may further include a bus 603. Theprocessor 601 and the non-transitory memory 602 may be connected to eachother through the bus 603. The bus 603 may be a Peripheral ComponentInterconnect (PCI) bus or an Extended Industry Standard Architecture(EISA) bus. The bus 603 can be divided into an address bus, a data bus,a control bus, and the like. For ease of representation, only one thickline is shown in FIG. 6, which does not mean that there is only one busor one type of bus. The mobile terminal 600 may further include inputand output devices 604, and the input and output devices 604 may includea display screen, such as a liquid crystal display. The non-transitorymemory 602 is configured for storing one or more programs includinginstructions. The processor 601 is configured to invoke the instructionsstored in the non-transitory memory 602 to perform some or all of themethod in FIGS. 1-5 described above. The mobile terminal 600 may alsoinclude N microphones and N speakers.

In the mobile terminal as shown in FIG. 6, during an audio recording,the reverberation algorithm may be determined according to relativelocation relationship of N sound sources, and a sound effect processingis performed for the N audio signals generated from the N sound sourcesaccording to the determined reverberation algorithm. It is achieved thatmultiple audio signals of multiple sound sources are performed with asound effect processing. A sound effect processing can be performed formultiple audio signals during a recording, so as to improve sound effectof an audio recorded in a multi-person audio recording scene.

Another mobile terminal is further provided in an embodiment of thepresent disclosure. As shown in FIG. 7, for the convenience ofdescription, only the parts related to the embodiments of the presentdisclosure are shown. If the specific technical details are notdisclosed, please refer to the method part of the embodiment of thepresent disclosure. The mobile terminal may be any terminal deviceincluding a mobile phone, a tablet computer, a PDA (Personal DigitalAssistant), a POS (Point of Sales), an in-vehicle computer, and thelike. The mobile terminal is used as a mobile phone as an example:

FIG. 7 is a block diagram showing a partial structure of a mobile phonerelated to a mobile terminal provided by an embodiment of the presentdisclosure. As shown in FIG. 7, the mobile phone includes a radiofrequency (RF) circuit 910, a memory 920, an input unit 930, a displayunit 940, a sensor 950, an audio circuit 960, a wireless fidelity(Wi-Fi) module 970, a processor 980, and a power supply 990. It will beunderstood by those skilled in the art that the structure of the mobilephone as shown in FIG. 7 does not constitute a limitation to the mobilephone, and may include more or less components than those illustrated,or some components may be combined, or different component arrangements.

The components of the mobile phone will be described in detail withreference to FIG. 7 in the following.

The RF circuit 910 can be used for receiving and emitting information.Usually, the RF circuit includes, but not limited to, an antenna, atleast one amplifier, a transceiver, a coupler, a Low Noise Amplifier(LNA), a duplexer and so on. In addition, the RF circuit may furthercommunicate with other devices via wireless communication and a network.The above wireless communication may use any communication standard orprotocol, including but not limited to Global System of Mobilecommunication (GSM), General Packet Radio Service (GPRS), Code DivisionMultiple Access (Code Division), Multiple Access (CDMA), Wideband CodeDivision Multiple Access (WCDMA), Long Term Evolution (LTE), E-mail,Short Messaging Service (SMS), and the like.

The memory 920 may be configured to store software programs and modules,and the processor 980 executes various function applications and dataprocessing of the mobile phone by running the software programs and themodules stored in the memory 920. The memory 920 may mainly include aprogram storage region and a data storage region, wherein the programstorage region may store an operation system, application programs forat least one function (for example, an audio playing function, an imageplaying function, etc.), and the data storage region may store data (forexample, audio data, telephone directory, etc.) created according to useof the mobile phone. In addition, the memory 920 may include ahigh-speed RAM, and may further include a non-volatile memory such as atleast one of a disk storage device, a flash device, or othernon-volatile solid storage devices.

The input unit 930 may be configured to receive input digital orcharacter information and generate key signal input associated with usersetting and functional control of the mobile phone. Specifically, theinput unit 930 can include a fingerprint identification assembly 931 andother input devices 932. The fingerprint identification assembly 931 cancollect data of a user's fingerprint. In addition to the fingerprintidentification assembly 931, the input unit 930 may also include otherinput devices 932. Specifically, the other input devices 932 mayinclude, but not limited to, one or more of a physical keyboard, afunctional key (such as a volume control key and a switch key), a trackball, a mouse, an operating rod, etc.

The display unit 940 can be used to display information input by theuser or information provided to the user as well as various menus of themobile phone. The display unit 940 may include a display screen 941,which is alternatively configured to be a Liquid Crystal Display (LCD),an Organic Light-Emitting Diode (OLED) and the like.

The mobile phone may also include at least one type of sensor 950, suchas a light sensor, motion sensor, and other sensors. Specifically, thelight sensor may include an ambient light sensor (also called lightsensor) and a proximity sensor, wherein the ambient light sensor mayadjust the brightness of the display screen 941 according to thebrightness of the ambient light, and the proximity sensor may turn offthe display screen 941 and/or backlight when the mobile phone is movedto the ear. As a motion sensor, the accelerometer sensor can detect themagnitude of acceleration in all directions (usually three axes). Whenit is stationary, it can detect the magnitude and direction of gravity,which can be used to identify the gesture of the mobile phone (such ashorizontal and vertical screen switching, related games, magnetometerattitude calibration), vibration recognition related functions (such aspedometer, tapping), etc. Other sensors such as gyroscopes, barometers,hygrometers, thermometers, infrared sensors can be configured on themobile phone, which are no longer described herein.

An audio circuit 960, a speaker 961 and a microphone 962 may provide anaudio interface between the user and the mobile phone. The audio circuit960 can convert the received audio data to the electrical data and thenemit to the speaker 961. The speaker 961 then converts to the soundsignal. On the other hand, the microphone 962 converts the collectedsound signal into an electrical signal. The audio circuit 960 receivesthe electrical signal and then convert it into audio data. The processor980 processes the audio data and then transmits another mobile phone viathe RF circuit 910, or transmits to the memory 920 for furtherprocessing.

Wi-Fi belongs to a short-range wireless transmission technology. Themobile phone may assist the user to receive and send e-mails, webpagebrowsing, access to streaming media and the like by means of the Wi-Fimodule 970, which provides a wireless wideband internet access for theuser. Although the Wi-Fi module 970 is illustrated in FIG. 7, it may beunderstood that, it may be optional components of the mobile phone andmay totally be omitted without changing the essence of the presentdisclosure as claimed.

The processor 980 is a control center of the mobile phone, which isconnected to all parts of the mobile phone by utilizing variousinterfaces and lines and execute various functions and processing dataof the mobile phone by running or executing the software program and/orthe module stored in the memory and calling data stored in the memory920. Thus, it wholly monitors the mobile phone. Optionally, theprocessor 980 may include one or more processing units. The processor980 may be integrated with an application processor and amodulation-demodulation processor. The application processor mainlyprocesses an operation system, a user interface, an application programand the like, and the modulation-demodulation processor mainly processeswireless communication. It will be appreciated that the above describedmodulation-demodulation processor may also not be integrated into theprocessor 980.

The mobile phone also includes a power supply 990 (such as a battery)that supplies power to the various components. Preferably, the powersupply can be logically coupled to the processor 980 through a powermanagement system to manage functions such as charging, discharging, andpower management through the power management system.

The mobile phone may further include a camera 9100. The camera 9100 isconfigured to capture an image or a video, and transmit the capturedimage and video to the processor 980 to be processed.

The mobile phone may further include a Bluetooth module, and the like,which are not described in detail herein again.

In the foregoing embodiments shown in FIG. 1 to FIG. 4, eachactions/operation of the method can be implemented based on thestructure of the mobile phone.

A computer storage medium is further provided in embodiments of thepresent disclosure. The computer storage medium stores computer programsfor electronic data exchange. The computer programs cause a computer toperform some or all of the actions of any of the processing methodsdescribed in the foregoing embodiments.

A computer program product is provided in embodiments of the presentdisclosure. The computer program product includes a non-transitorycomputer readable storage medium storing a computer program. Thecomputer programs cause a computer to perform some or all of the actionsof any of the processing methods described in the foregoing embodiments.

It should be noted that, for the foregoing method embodiments, for thesake of brevity, they are all described as a series of actioncombinations, but those skilled in the art should understand that thepresent disclosure is not limited by the described action sequence.Because certain steps may be performed in other sequences orconcurrently in accordance with the present disclosure. In thefollowing, those skilled in the art should also understand that theembodiments described in the specification are all preferredembodiments, and the actions and modules involved are not necessarilyrequired by the present disclosure.

In the above embodiments, the descriptions of the various embodimentsare all focused on, and the parts that are not detailed in a certainembodiment can be referred to the related descriptions of otherembodiments.

In several embodiments provided herein, it should be understood that thedisclosed apparatus may be implemented in other ways. For example, thedevice embodiments described above are merely illustrative. For example,the division of the above units is only a logical function division. Inactual implementation, there may be another division manner. Forexample, multiple units or components may be combined or integrated toanother system, or some features can be ignored or not executed. Inaddition, the mutual coupling or direct coupling or communicationconnection shown or discussed may be an indirect coupling orcommunication connection through some interface, device or unit, and maybe electrical or otherwise.

The units described above as separate components may or may not bephysically separated, and the components illustrated as units may or maynot be physical units, which may be located in one place, or may bedistributed to multiple network units. Some or all of the units may beselected according to actual needs to achieve the purpose of thesolution of the embodiment.

In addition, each functional unit in each embodiment of the presentdisclosure may be integrated into one processing unit, or each unit mayexist physically separately. Alternatively, two or more units may alsobe integrated into one unit. The above integrated unit can beimplemented in the form of hardware or in the form of a softwarefunctional unit.

The above-described integrated unit can be stored in a computer readablememory if it is implemented in the form of a software functional unitand sold or used as a standalone product. Based on such understanding,the technical solution of the present disclosure, in essence or thecontribution to the prior art, or all or part of the technical solutionmay be embodied in the form of a software product. The software productis stored in a memory, which includes a number of instructions causing acomputer device (which may be a personal computer, server or networkdevice, etc.) to perform all or part of the actions of theabove-described methods of various embodiments of the presentdisclosure. The foregoing memory includes a U disk, a Read-Only Memory(ROM), a Random Access Memory (RAM), a removable hard disk, a magneticdisk, or an optical disk, and the like, which can store program codes.

One of ordinary skill in the art can understand that all or part of thevarious methods of the above embodiments can be completed by relatedhardware instructed by a program. The program can be stored in acomputer readable memory, and the memory can include a flash drive,read-only memory (ROM), random access memory (RAM), disk or CD.

The embodiments of the present disclosure have been described in detailabove, and the principles and implementations of the present disclosureare described in the specific examples. The description of the aboveembodiments is only used to help understand the method of the presentdisclosure and its core ideas. For a person skilled in the art, therewill have a change in the specific embodiments and the scope of presentdisclosure according to the idea of the present disclosure. In summary,the content of the present specification should not be construed aslimiting the present disclosure.

What is claimed is:
 1. A method of processing for sound effect ofrecording, comprising: recording N audio signals generated from N soundsources, N being an integer greater than or equal to 2; obtaininglocation relationships of the N sound sources, determining a scene wherethe N sound sources are located in, and determining a reverberationalgorithm according to the location relationships of the N sound sourcesand the scene where the N sound sources are located in; and performing asound effect processing for the N audio signals according to thereverberation algorithm to obtain a reverbed sound effect; wherein thescene where the N sound sources are located in comprises an indoorscene; and the determining the reverberation algorithm according to thelocation relationships of the N sound sources and the scene where the Nsound sources are located in comprises: determining indoor parameters ofthe indoor scene, wherein the indoor parameters comprise a parameter ofindoor area, a parameter of indoor volume, a parameter of indoormaterial, and a parameter of indoor air medium; determiningreverberation parameters according to the indoor parameters and thelocation relationships of the N sound sources, wherein the reverberationparameters comprise a ratio of direct sounds, attenuation coefficientscorresponding to the direct sounds, a radio of echoes, attenuationcoefficients corresponding to the echoes, and a reverberation frequencyrange; and determining the reverberation algorithm according to thereverberation parameters.
 2. The method of claim 1, wherein recordingthe N audio signals generated from the N sound sources comprises:recording the N audio signals generated from the N sound sources throughN microphones, each of which is arranged adjacent to a respective one ofthe N sound sources.
 3. The method of claim 2, wherein obtaining thelocation relationships of the N sound sources comprises: measuring adistance and a relative orientation between any two of the N microphonesthrough a measuring device.
 4. The method of claim 3, whereindetermining the reverberation algorithm according to the locationrelationships of the N sound sources and the scene where the N soundsources are located in comprises: determining the reverberationalgorithm according to the distance and relative orientation between anytwo of the N microphones and the scene where the N sound sources arelocated in.
 5. The method of claim 1, wherein the method is applied to avideo call terminal; and obtaining the location relationships of the Nsound sources comprises: obtaining a first location of each of the Nsound sources with respect to the video call terminal.
 6. The method ofclaim 5, wherein determining the reverberation algorithm according tothe location relationships of the N sound sources and the scene wherethe N sound sources are located in comprises: determining thereverberation algorithm according to the first location of the each ofthe N sound sources with respect to the video call terminal and thescene where the N sound sources are located in.
 7. The method of claim1, wherein performing the sound effect processing for the N audiosignals according to the reverberation algorithm to obtain the reverbedsound effect comprises: performing a denoising processing for the Naudio signals to obtain the denoised N audio signals through amulti-noise filtering engine; and performing the sound effect processingfor the denoised N audio signals according to the reverberationalgorithm to obtain the reverbed sound effect.
 8. The method of claim 1,further comprising: transmitting the reverbed sound effect to a videocall terminal to play the reverbed sound effect.
 9. A mobile terminal,comprising a processor and a non-transitory memory storing one or moreprograms, wherein the one or more programs are configured to be executedby the processor to perform a method, and the method comprises:recording N audio signals generated from N sound sources, N being aninteger greater than or equal to 2; obtaining location relationships ofthe N sound sources, determining a scene where the N sound sources arelocated in, and determining a reverberation algorithm according to thelocation relationships of the N sound sources and the scene where the Nsound sources are located in; and performing a sound effect processingfor the N audio signals according to the reverberation algorithm toobtain a reverbed sound effect; wherein the scene where the N soundsources are located in comprises an indoor scene; and the reverberationalgorithm is determined according to reverberation parameters, whereinthe reverberation parameters are determined according to indoorparameters of the indoor scene and the location relationships of the Nsound sources and comprise a ratio of direct sounds, attenuationcoefficients corresponding to the direct sounds, a radio of echoes,attenuation coefficients corresponding to the echoes, and areverberation frequency range, and the indoor parameters comprise aparameter of indoor area, a parameter of indoor volume, a parameter ofindoor material, and a parameter of indoor air medium.
 10. The mobileterminal of claim 9, wherein each of the N audio signals are recordedthrough a respective one of N microphones, and each of the N microphonesis arranged adjacent to a respective one of the N sound sources.
 11. Themobile terminal of claim 10, wherein the location relationships of the Nsound sources comprise a distance and a relative orientation between anytwo of the N microphones.
 12. The mobile terminal of claim 11, whereinthe reverberation algorithm is determined according to the distance andthe relative orientation between any two of the N microphones and thescene where the N sound sources are located in.
 13. The mobile terminalof claim 9, wherein the location relationships of the N sound sourcescomprise a first location of each of the N sound sources with respect tothe mobile terminal.
 14. The mobile terminal of claim 13, wherein thereverberation algorithm is determined according to the first location ofthe each of the N sound sources with respect to the mobile terminal andthe scene where the N sound sources are located in.
 15. A non-transitorystorage medium storing one or more programs, when executed, causing aprocessor to perform a processing method, wherein the processing methodcomprises: recording N audio signals generated from N sound sources, Nbeing an integer greater than or equal to 2; obtaining locationrelationships of the N sound sources, determining a scene where the Nsound sources are located in, and determining a reverberation algorithmaccording to the location relationships of the N sound sources and thescene where the N sound sources are located in; and performing a soundeffect processing for the N audio signals according to the reverberationalgorithm to obtain a reverbed sound effect; wherein the scene where theN sound sources are located in comprises an indoor scene; and thereverberation algorithm is determined according to reverberationparameters, wherein the reverberation parameters are determinedaccording to indoor parameters of the indoor scene and the locationrelationships of the N sound sources and comprise a ratio of directsounds, attenuation coefficients corresponding to the direct sounds, aradio of echoes, attenuation coefficients corresponding to the echoes,and a reverberation frequency range, and the indoor parameters comprisea parameter of indoor area, a parameter of indoor volume, a parameter ofindoor material, and a parameter of indoor air medium.
 16. Thenon-transitory storage medium of claim 15, wherein each of the N audiosignals are recorded through a respective one of N microphones, and eachof the N microphones is arranged adjacent to a respective one of the Nsound sources.
 17. The non-transitory storage medium of claim 16,wherein the location relationships of the N sound sources comprise adistance and a relative orientation between any two of the Nmicrophones.